# 



# 



Attorney Docket No. 9432-000089 



CONTENT-BASED AUTHENTICATION 
OF GRAPH PRESENTED IN TEXT DOCUMENTS 



BACKGROUND OF THE INVENTION 



Technical Field 

The present invention relates generally to document authentication. 
More particularly, the present invention relates to the authentication of graphs 
5 at the object level as well as the pixel level. 

Discussion 

For as long as humans have communicated with one another, there has 
been concern over maintaining confidentiality. As a result, verbal, written, and 
electronic messages have all been the subject of substantial technological 

10 efforts to maintain security. For example, document authentication techniques 
are commonly used to ensure the integrity of a wide variety of electronic 
documents such as, presentations, contracts, military orders, and databases. 
Authentication involves the task of making the determination that the 
document has not been tampered with and that it originated with the 

15 presumed transmitter. Authentication using digital watermarks is a particular 
technique that has been studied by many researchers in the last ten years. 
For example, digital watermarking has been successfully applied to digital 
documents such as digital color/gray scale images and plain text. While 
electronic document authentication efforts have experienced considerable 
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success, it is important to note that these efforts have typically centered around 
the protection of textual documents and images. 

Recently, however, more and more documents are using graphs in 
addition to images and text for system and idea illustration. In contrast to 
5 images, graphs are more difficult to watermark because of low capacity of 
additive noise. This is due to the binary nature of graphs. The term "binary 
nature" relates to the fact that most graphs have one bit per pixel, whereas 
most images have multiple bits per pixel to 'indicate varying shades and 
colors. Binary pixels make it particularly hard to insert watermarks due to the 

10 low capacity for perceptual invisible noise. In other words, a minimal 
alteration of bits in a binary graph can result in a substantial change in the 
appearance and content of the graph. Furthermore, the critical information of 
a graph is often contained at the object level rather than the pixel level. For 
example, a useful application for document copying and copyright protection 

is is to provide different levels of access to different users. In such a case it 
would be very desirable to detect alteration of the original document as well 
as localize the alteration on the object level. For example, it is more 
important to detect a substantive change in a document, such as "10%" to 
"70%", than it is to detect an increase in the size of an arrow by one pixel. 

20 Thus, the sensitive information in a document is often contained on the object 
level rather than the pixel level. 



-2- 



Attorney Docket No. 9432-000089 

Pixel level authentication may also result in less flexibility. For 
example, if the annotation font of a graph changes but the content of the 
graph does not, pixel level authentication will alert the owner that the 
annotations have been altered. The owner has no way of determining, 
5 however, that the content of the graph matches the original. Object level 
authentication, on the other hand, would assure the owner that the "content is 
authentic" in such a case. If the font is marked as sensitive information, 
object level authentication could also alert the owner to font alterations. In 
many applications, however, it would be highly desirable to provide a 

10 mechanism for returning an "authentic" determination if the font is not marked 
as sensitive information. 

Conventional methodologies for content-based text authentication 
mainly rely on altering the word/line spacing or the length of character vertical 
serif strokes. While text documents are often referred to as binary images 

15 and share the same binary nature of graphs, these methodologies can hardly 
be extended to authentication of graphs. This is because even on the pixel 
level graphs generally do not exhibit the same characteristics as text. For 
instance, in a graphical flowchart the shape of each node may be very 
important, whereas the nodes often have substantially fewer characters as 

20 compared to a paragraph of text. In such a flowchart the number of objects 
that exhibit a vertical serif can be as low as a few percent of the total number 
of objects. Here, an object is referred to an alterable line, character, or curve. 
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In fact, other kinds of graphs may not exhibit alterable line spacing or vertical 
serif at all. It is therefore desirable to bridge text-based authentication 
techniques to the authentication of graphs. 

SUMMARY OF THE INVENTION 

5 The above and other objects are provided by a computerized method 

for authenticating a document. The method includes the step of partitioning 
the document into graphical content and textual content. The graphical 
content is then converted into a symbolic representation of the graphical 
content. The method further provides for authenticating the symbolic 
10 representation with a text authentication algorithm. 

The present invention also provides a computerized method for 
authenticating a binary graph. The graph is authenticated at the pixel level 
as well as the object level. The method includes the step of encrypting the 
authenticated graph, 

15 As a further aspect of the invention, a graph authentication system has 

an object level authenticator for authenticating a graph at an object level. 
The authentication system further includes a pixel level authenticator for 
authenticating the graph at a pixel level and an encryption system for 
encrypting the authenticated graph. 

20 It is to be understood that both the foregoing general description and 

the following detailed description are merely exemplary of the invention, and 
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are intended to provide an overview or framework for understanding the 
nature and character of the invention as it is claimed. The accompanying 
drawings are included to provide a further understanding of the invention, 
and are incorporated in and constitute part of this specification. The 
5 drawings illustrate various features and embodiments of the invention, and 
together with the description serve to explain the principles and operation of 
the invention. 

BRIEF DESCRIPTION OF THE DRAWINGS 

The various advantages of the present invention will become apparent to 
10 one skilled in the art by reading the following specification and appended claims, 
and by referencing the following drawings in which: 

Figure 1 is a block diagram of a graph authentication system according to 
the present invention; 

Figure 2 is a block diagram of an object level authenticator according to 
15 the present invention; 

Figure 3 is a block diagram of a pixel level authenticator according to the 
present invention; 

Figure 4 is a flowchart of a computerized method for authenticating a 
document according to the present invention; 
20 Figure 5 is a flowchart of the process of authenticating a graph at the 

object level according to the present invention; 
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Figure 6 is a flowchart of the process for authenticating a graph at the 
pixel level according to the present invention; 

Figure 7 is a flowchart for the process of adding visible authorization 
information according to the present invention; 
5 Figure 8 is a flowchart for the process of adding invisible authorization 

information according to the present invention; 

Figure 9 is a sample illustration of graphs which can be authenticated with 
the present invention; 

Figure 10 is an illustration of a graphical flowchart which can be 
10 authenticated with the present invention; 

Figure 11 is a block diagram of a one-party owned document 
authentication process according to the present invention; 

Figure 12 is an illustration of a key set according to the present invention; 

Figure 13 is a table of relationship and specification symbols according to 
15 a preferred embodiment of the present invention; 

Figure 14 is a symbolic representation of the graphical flowchart of Figure 

10; 

Figure 1 5 is a graphical flowchart authenticated at the pixel level using a 
bounded box according to the present invention; 
20 Figure 16 is a graphical flowchart authenticated at the pixel level using a 

bar code according to the present invention; 

Figure 17 is an enlarged view of textual and graphical content containing 
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invisible authentication information; and 

Figure 18 is a table comparing graph authentication algorithms. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 

Turning now to Figure 1, the preferred embodiment of the graph 
5 authentication system 20 includes an object level authenticator 30, a pixel 
level authenticator 40, and an encryption system 50. The graph 
authentication system 20 provides for content-based authentication of graphs 
contained in a host document 51. The result is protected document 52. As 
part of the following discussion, I is defined to be the host document 51, such 

io as a contract, which will be authenticated by owner 01 or owners 01, 02 to 
On. The authenticated copy of host document I is denoted as 7 . In 
correspondence, G and G are defined to be the original and the 
authenticated copy of a graph respectively. Furthermore, R is defined as an 
authorized receiver, whereas A is an attacker, i.e., unauthorized receiver. 

15 The following scenarios illustrate potential applications and objectives of 
graph authentication system 20. 

The first scenario is the situation in which h g 01, 01 wants to determine 
whether her document h is authentic. The content of the document contains 
sensitive information, such as a price of $1,000 or a deadline of June 01, 1999. 

20 Another scenario occurs when U e 01, 01 needs to send h to R and wishes to 
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grant R "read" permission but not "write" permission. A variation on this scenario 
is the situation in which 01 wants to prevent alteration of any kind and to localize 
the alterations made by an attacker A who gets h from 01 and then sends it to R. 
Or, 01 may want everyone to be able to read U while only herself and R can 
5 make modification on the document. Another scenario occurs when h g O1o 
02, i.e., h is a contract between 01 and 02. If the copy in Ol's hand is different 
from that of in 02 s, 01 wants to prove that 02's copy is a tampered copy of the 
original contract by checking the authenticity of 02 s copy. In addition, 01 may 
want to point out where exactly 02 altered the original contract. 

10 Turning now to Figure 11, it can be appreciated that the present 

invention provides a fully functional content-based authentication system for 
text documents including binary graphs. By building a bridge from graph to 
text on the character level, the present invention allows authentication of 
graphs using suitable text document authentication algorithms. When pixel 

is level precision of a graph is required, a pixel level authentication can be 
added. This layer lets the owner detect as well as localize changes in the 
graph on the pixel level. The hierarchical layout allows the application of the 
present invention to the aforementioned scenarios as well as other scenarios. 
The first level of the hierarchy is the pixel level authentication which is 

20 followed by an object level authentication. These are done with owner Ofs 
private key. Notice here, either the pixel level or the object level protection is 
optional depending on the application. For ultimate protection, however, a 
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dual-layer protection with a pixel protection layer plus an object protection 
layer is recommended since the two layers are orthogonal. Additionally, a 
meaningful watermark, such as a company logo, can be inserted, if desirable, 
into the document. Furthermore, the authenticated documents, including text 
5 and graphs can be encrypted with a public key encryption algorithm for 
secure transmission. Here the watermarking layer can be done either before 
or after the authentication layer. This again, depends on different 
applications. Access authorization can then be granted by distributing 
different keys to different users. For example, in the case of "read" only 

10 access, R will be given the public decryption key K 4 only. In the case of a 
multi-party owned document authentication, each party has a private key, the 
authentication is done by generating a key set with the private key from every 
party (see Figure 12). Attempted modifications of the document without a key 
will therefore be unsuccessful. 

15 Returning to Figure 1, it will be appreciated that the object level 

authenticator 30 authenticates the graph at an object level, whereas the pixel 
level authenticator 40 authenticates the graph at a pixel level. The encryption 
system 50 encrypts the authenticated graph for transmission to the recipient. As 
seen in Figure 2, it will be appreciated that the object level authenticator 30 

20 converts the graph into a symbolic representation of the graph via a specification 
module 31 and a relationship module 32. The specification module 31 defines 
nodes of the graph with specification symbols. Similarly, the relationship module 
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32 defines relationships between the nodes of the graph with relationship 
symbols. This allows a text authentication module 33 to authenticate the 
symbolic representation with a text authentication algorithm. 

Figure 9 demonstrates the various types of graphs which can be 
5 authenticated via the present invention. The operation of the object level 
authenticator 30 can be better understood through the graphical flowchart of 
Figure 10. It can be appreciated that the important information contained in 
graphical flowchart 34 is the annotation of each node and the connections 
between nodes that illustrate the relationship of nodes. Whether the drawing of 

10 each box is slightly smaller or slightly larger, the length of a line is longer or 
shorter, or the position of a node is tilted to the left or right is generally not as 
important. Consequently, the authentication process is mainly concerned with 
the object level instead of the pixel level of the graphical flowchart 34. It is 
important to note that the important characteristics of an object depend on the 

15 type of graph. Thus, in the case of the bar chart of Figure 9(c), the important 
information is contained in the relative height of each individual bar rather than 
the overall height of the graph. For example, if the height of the second bar is 
changed to half its original height, the value of the second bar is thereby altered. 
It will be appreciated that the concern with most text documents is at the object 

2 o level, or character level. 

Graphical flowchart 34 therefore includes various nodes and lines and 
can be represented with a series of relationship symbols along with the node 
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annotations as follows: "<Ni{'Process A\ #1, &reg, @mid} N 2 {'Process B', #1, 
&reg, @mid}-> N 3 {'lf C\ #3, &reg, @mid}-»< N 4 {'End', #2, &reg, @mid}|yes; 
N2|no»", wherein Figure 13 illustrates the relationship and specification symbols 
for the above symbolic representation. The result is shown in Figure 14. In the 
5 above symbolic representation, Ni N2... are node names with the property of 
each node contained in {}, <> is a tuple, and and | are relationship symbols. It 
will be appreciated that the properties of nodes and lines, the shape, size, color, 
and position, can be described with the specification symbols. For those 
specification insensitive graphs, the symbols between each pair of {} can be 

10 simply ignored whereas in specification sensitive graphs, the specification 
symbols in each pair of {} provide different levels of details. This hierarchical 
representation provides additional flexibility. 

After defining the nodes of the graph with specification symbols, and 
the conditions and familial relationships with relationship symbols, the text 

15 authentication module 33 can authenticate the symbolic representation. For 
example, well known two- or multi-dimensional checksum techniques can be 
used to verify authenticity. For the following discussion, let T(p,q) represent 
the (p,q)th character. S(p,q) = s 1 (p,q) s 2 (p,q) s J (p,q) = f(T(p,q)) is the coded 
representation of T(p,q) via map f, wherein s 1 (p,q) s 2 (p,q) "s J (p,q) represent 

20 the first, the second, ... and the Jth bit of S(p,q) that are in the order of the 
most significant bit to the least significant bit. Furthermore, let Sum P ,= £ P p=i 
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s j (p,q) and Sum q - £ Q q =i si (P.q). where P & Q are dimensional sizes. Thus, the 
position (p,q) of any alteration Suny* Sum P , Surr^* Sum q can be localized. 

It will be appreciated that utilizing well known content-dependent one 
way hash functions provides a higher level of security. For the following 
5 discussion, let B denote the block size and K denote a private key. In the 
case of a multi-party document, K is a function of K01, K02, i.e., K = f1(Koi, 
Ko2, ...). Figure 12 illustrates a key set for the present example. For the 
purpose of discussion, we may assume each key in the set, Koi, Ko2, ... to be 
encrypted with its owner's private key, and an arbitrator (a trusted third party) 

10 is used to generate the key set K. It is important to note, however, that other 
suitable cryptography protocols may also be used. Assume K is a Jbits 
coding with the 1 st to (J-1) th bits being the code bits and the lowest bit, J th bit, 
being the verification bit. The document paragraph I shown in Figure 14 can 
use 9bits coding. Choosing the one way hash algorithm MD5, the encoding 

15 procedure is as follows. Pad the source text I to an exact multiple of 512 in 
length. For each 128-length set, l 0) choose its neighborhood set, ]o=512 
characters with l 0 c= io. Assume 
So ={So(i), i€[1,128]} = {s 1 0 (i) s 2 0 (i)- s J 0 (i)}= f(U) 
and 

2 0 So ={So(i)> ,512]} = {slo(i) £o(\) So(\)} = f(]o) 
are coded representation of U and U respectively. 

1 . Concatenate the code bits of the neighborhood set U, 
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2. Calculate the 128bits hash value of it, ho=H(So), 

3. Generate message ho'=Sgn(K, h G ) by signing h Q with public 
cryptography method, and 

4. Put ho' into the J th bit, the lowest bit, of S 0 (i), i.e., let s J 0 (i)= h 0 '(i), 
5 ie[1,128]. 

The above algorithm is discussed in the context of image 
authentication in the article "Fragile imperceptible digital watermark with 
privacy control", C. W. Wu, D. Coppersmith, F. C. Mintzer, C. P. Tresser, and 
M. M. Yeung, IS&T/SPIE Conference on Security and Watermarking of 

10 Multimedia Content, SPIE 3657, Jan, 1999, incorporated herein by reference. 
The decoding process is similar to the encoding process with the verification 
done through an XOR operation. Such that Auth G (i)= K '(i)© s J 0 (i). 
If Auth 0 (i)=1 for vie[1,128], the l 0 set has been altered. 
Turning now to Figure 3, the pixel level authenticator 40 of the graph 

15 authentication system 20 is shown in greater detail. It can be appreciated 
that a visible watermarking module 41 adds visible authentication information 
to the graph at the pixel level, whereas an invisible watermarking module 42 
adds invisible authentication information to the graph at the pixel level. The 
preferred embodiment further includes a coalescing module 43 for embedding 

20 a hash value from the object level of the graph at the pixel level of the graph. 
Dual level authentication with coalescing has been found to yield optimum 
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results. To authenticate I with N symbols, we compute the one way hash of I 
on the character level first. Therefore, if N=248 characters this is done by 
putting all the bits of the 248 characters together, pad the result to an exact 
multiple of 512 in length, and calculate the hash value of the padded 
5 message. Then, the 128bits hash value is embedded at the pixel level. 

Operation of the graph authentication system of the present invention will 
now be described in greater detail for programming purposes. Turning to Figure 
4, a computerized method for authenticating an electronic file (or document) is 
shown generally at 100. Step 102 demonstrates receipt of the electronic file. At 

10 step 101, the file is partitioned into graphical content and textual content. The 
partitioning of graphs from text regions in a document has been the subject of 
considerable study. For example, U.S. Patent No. 5,465,304, and U.S. Patent 
No. 5,335,290 to Cullen, et al., incorporated herein by reference, discuss the 
segmentation of text, pictures, and lines of a document image. Furthermore, U.S. 

15 Patent No. 5,073,953 to Westdijk, incorporated herein by reference, discloses a 
system and method for automatic document segmentation. The separation of 
body text from other regions of a document is taught in U.S. Patent No. 
5,892,843 to Zhou, et al., incorporated herein by reference. Also, in U.S. Patent 
No. 5,379,130 to Wang, et al., a method and system that separates images from 

20 text is disclosed. Any of these techniques or other well known approaches can 
be readily adapted to perform partitioning step 101. 
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At step 110, it is determined whether the object level is a level of concern. 
If so, the graph is authenticated at the object level at step 111 by adding 
authentication information the electronic file based on an object level 
representation. Similarly, at step 130 it is determined whether the pixel level is a 
5 level of concern. If so, the document is authenticated at the pixel level at step 
131. It will be appreciated that object level authentication and pixel level 
authentication are both optional and can be performed in any order. The graph 
can then be encrypted at step 150 and transmitted at step 160 to an authorized 
recipient. 

io Figure 5 shows step 111 in greater detail. It can be appreciated that 

nodes of the graph are defined with specification symbols at step 112. 
Relationships between the nodes are then defined with relationship symbols at 
step 113. The result is a symbolic (or object level) representation of the 
graphical content contained in the electronic file. It will be appreciated that other 

15 approaches to object level representation can be taken without parting form the 
scope of the invention. At step 1 14, the symbolic representation is authenticated 
.with a text authentication algorithm. 

Turning now to Figure 6, step 131 is shown in greater detail. At step 132, 
it is determined whether transparency is required based on the content of the 

20 graph and the host document. If so, invisible authorization information is added 
at step 133. Otherwise, visible authorization information can be added at step 
134. 
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As seen in Figure 7, a relatively robust approach for adding visible 
authorization information is shown in greater detail. Specifically, at step 135 
a truncated image of the graph is formed. For the following discussion, let 
XxY=128 be the defined block size. Graph G can therefore be cut into XxY 
5 blocks. Assuming the number of blocks is L, we concatenate the bits of the 
(x,y)th pixel of every block to the 1 st block and form an Lbits truncated image 
TrunG. Therefore, a Lbits/pixel image TrunG, with image size XxY, of graph 
G is generated. Let TrunG(x,y)' denote the t bit of pixel (x t y) of TrunG. 
Notice here, it is desirable to form the truncated image TrunG in such a way 
10 that TrunG(x,y)^0. Also note that to get a higher level of protection, a random 

number generator should be used to cut the graph. 

At step 1 36, an initial message is generated from the truncated image. 

The initial message is defined by all bits of the truncated image. Thus, step 136 

collects all bits of all XxY pixels into a XxYxL bits message M1. At step 137, the 
15 initial message is converted into a padded message, wherein the padded 

message has a size defined by a multiple of a predetermined length. Thus, M1 is 

padded into an exact multiple of 512 in length with as many zeros as needed to 

obtain message M1\ 

At step 138, a hash value for the padded message is computed. Thus, 
2 0 step 138 computes the 128 bits hash value of MV using MD5, M2=h(l)=H(M1'). 

At step 139, the hash value is converted into a public key encrypted message by 
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signing the hash value with a public key cryptography method such that 
M3=h'(i)=Sgn(K ) M2). The public key encrypted message is then converted into 
visible authentication information at step 140. The visible authentication 
information can be in many different formats. For example, Figure 15 illustrates 
5 an authenticated graph using a bounding box, whereas Figure 16 illustrates an 
authenticated graph using a bar code. 

When invisible authentication is required or desirable, a less robust 
scheme that modifies the graph itself can be used. Thus, as shown in Figure 8, a 
truncated image is formed from the graph at step 135'. At step 141 , a verification 

10 bit is selected from each pixel of the truncated image. Thus, at step 141 1 bit 
TrunG(x,y)' =1 out of the Lbits of each pixel (x,y) in TrunG to be the verification 
bit. For better imperceptibility and a higher lever of security, the verification bits 
should be picked in a way to maximize spread. 

At step 136' an initial message is generated from the truncated image, 

15 wherein the initial message is defined by all non-verification bits of the truncated 
image. Step 136' therefore collects the remaining (L-1) bits of all XxY pixels into 
a XxYx(L-1) bits message M1. Message M1 is padded into an exact multiple of 
512 in length with as many 0s as needed and get message M1\ The initial 
message is therefore converted into a padded message at step 137'. Preferably, 

20 the padded message has a size defined by a multiple of a predetermined length 
of 512. 



- 17 - 




Attorney Docket No. 9432-000089 

At step 138\ the hash value is computed for the padded message. The 

hash value is then converted into a public key encrypted message at step 139'. 

The public key encrypted message can then be imbedded into the truncated 

image at step 142 in the following fashion: 

5 . |f h'(i)=h'((y-1 )*X+x))=0 and |TrunG(x,y)|= odd, let TrunG(x,y)=0. 

- If h , (i)=h , ((y-1)*X+x))=1 and |TrunG(x,y)|= even, let TrunG(x,y)'=1. 

Where |TrunG(x,y)| denotes the cardinality of TrunG(x t y), i.e., the number 
of bits that are '1 's among the Lbit of TrunG(x f y). 

io Turning now to Figure 17, two sample results can be seen. The lower 

result is cropped from the graphical flowchart in Figure 10. To give a better view, 
each result is enlarged to at least 400 percent of the original size. 

Conventional space-shifting methods and serif-modification methods are 
proposed in "Electronic Marking and Identification Techniques to Discourage 

15 Document Copying", J. Brassil, S. Low, N. Maxemchuk, and L. O'Gorman, IEEE 
Infocom 94, and in "Document Marking and Identification using Both Line and 
Word Shifting", S. H. Low, N. F. Maxemchuk, J. T. Brassil, and L. O'Gorman, 
Infocom '95, both incorporated herein by reference. Comparing these techniques 
to the present invention, it can be seen in Figure 18 that clear improvement has 

20 been achieved. Notice that when the hash value is prepended to the document, 
special coding is not needed for object level authentication. Otherwise, such 
coding is needed. Similarly, in the case of pixel level or coalesced 
authentication, special coding is not needed with visible authentication 
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information, whereas it is needed for invisible authentication information. Here, 
special coding means a new code other than commonly accepted codes, such as 
ASCII Code and Unicode. 

The foregoing discussion discloses and describes exemplary 
5 embodiments of the present invention. One skilled in the art will readily 
recognize from such discussion, and from the accompanying drawings and 
claims, that various changes, modifications and variations can be made therein 
without departing from the spirit and scope of the invention as defined in the 
following claims. 



10 
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